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) In connection with an information processing netw«»k (16) In whfch multiple pressing devices 
(18.20 22 24) have individual cache memories and also share a main storage memory(28)^ process is 
disclosed' for allocating multiple date operations or tasks for subsequent execution by the Proce^ing 
de^ A plurality of task dispatching elements (TDE) {96-108) formig a task l^F^-JSJ^w^^ 
92 are scanned in an order of descending priority, for either a speciffc affim^ ^ ?J^J^^ «1 
orocessing devices, or a general affinity to all of the processing devices. TDEs wjthspecifte 
aSS bw^dl^^^ If the selected picessor is available, whfle TDEs of general affinity are «sa^. 
TD^ with a specific affinity are bypassed if the selected processor is not avaflable. or resenred if a 
predetermined^ass threshold has been reached. FoUowing the primary scan a ^^^^^J^J^^^ 
Sder ^^ndS priority, assigns any reserved tesks to the processing devias 

tolSoe^r affin^^ Previously bypassed tesks can be assigned as m ^^e^rt^ aiV 
Sc^ remains availabte. A further feature of the network Is a means to reset the PJ?'^^,^^^^ 
r^e^ task from the specific affinity to the general affinity. Resetting ^^^^^^^^^ 
assembly level instructton contained in the task, and either can be unconditional wittire^ ^S^i 
^^^tevo- the tesk is executed on one of the processing devices, or can occur only upon the fa8ure to 
meet a predetenmined conditton whBe the task is executing. 
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The present invention relates to information pro- 
cessing systems in which multiple processing devices 
with cache memories are coupled to a shared main 
storage memory, and more particularly to a process 
for allocating tasks or operations among the proces- 
sors. 

In data processing systems employing two or 
more processing devices, in which multiple taste or 
operations on data are eligible to be perfonmed or 
executed by more than one of the processing devices, 
there is a need to allocate the various tasks among the 
processing devtees. It is advantageous to execute the 
various tasks in a manner which fevers the more 
important or higher priority taste for eailier perform- 
ance. One approach to meeting these requirements is 
a task dispatching apparatus for an asynmnetric or 
symmetric multiprocessing system, disclosed in U.S. 
Patent Ho. 4,394,727. incorporated in this application 
by reference. A task dispatching queue (TDQ) 
includes a collection of task dispatching elements 
(TDE), each such element associated with a task and 
signifying the priority of its associated task. Each of 
the processors has a signal dispatcher that examines 
the TDQ to determine if tiie processor is running a 
task having a lower prmrity than any task in the queue, 
as indicated by the task dispatching element If so, a 
task dispatcher of the processor switches to the 
higher priority task. Thus, the Hoffman system con- 
tinually favcMTS the highest priori^ taste for earliest 
execution. 

U.S. Patent No. 4,796.178 Is drawn to a task con- 
trol mechanism for maintaining a queue of available 
processes associated accoatiing to an assigned 
priority for a piuralHy of central processors. Each of 
the processors may be assigned to the highest priority 
task available when not executing a task of higher 
priority. A processor identification is employed to pre- 
vent other processors from gaining access to the 
same task. On U.S. Patent Ho. 4,807,281, a processor 
can store a protective code corresponding to its 
address at a location associated with a particular job 
or task, but not if anottier processor already has 
entered its code at the job. A processor, upon detect- 
ing the protective code of another processor 
associated witii the task data. fonAfards its own task to 
the other processor to reroute the job, to prevent a 
single job from running on more than one processor 
simultaneously. 

In certain multiprocessor systems, each of the 
processors not only shares a common main storage 
memory, but has its own cache memory. This feature 
can substantially enhance network performance, 
since eadh time one of the processir^ devices per- 
forms a date store or date fetch operation in ite 
assodated cache menusry rather than in the shared 
main memory, memoiy access time is substentiaHy 
reduced. 

When a task is executing on a particular one of 



the processing devices, the cache memory of that pn>> 
cessing device accumulates working date and 
in^aicttons pertinent to that task as the task ccm- 
tinues to execute. In other words, the task develops a 

s natural affinity to the particular processing devtee. 
However, once the tesk is switched out of that pro- 
cessor. Its natural affinity for that processor tends to 
decay, for either of two reasons. First, sut>sequent 
teste executing on that processor enter their own date 

10 into the cache memory, replacing date pertinent to the 
original tesk. Alternatively, ttie original task may exeo- 
ute, tHJt on another processor. Date stores performed 
in the course of such further execution invalidate cor- 
responding date in the original cache memory. 

15 Accordingly, a tesk usually has the strongest natural 
affinity for the processing device which has most 
recentiy performed or executed the tesk. 

In view of the atMve. it is not surprising in that in 
multiprocessor infbnnation systems with individual 

20 cache memories, performance is improved when 
teste are selectively allocated to processors for which 
they have affinity. Such a tesk allocation, however, 
firequentiy competes with allocation based on task 
priority, which of course contemplates assigning the 

25 highest priority tesk to the first available processor, 
without regard for affinity. 

Therefore, it is object of the present invention to 
provide a tesk allocation system capable (tf t>alancing 
the competing objectives of executing tasks on pro- 

30 cessors for which they have the greatest aHonify, 
favoring tiie highest priori^ teste, and provMing a 
substentiaHy equal processing load to all processing 
devices. 

Another object of the invention is to provide a 
35 means for associating taste witti particuter proces- 
sors In a mannerthat approximates theirnatural aff^i- 
ties. 

A further object is to provide a system fox allocat- 
ing tesks based on processor affinity, with means for 
40 preempting affinity-related allocation in response to 
detecting an imbalance of work load among proces- 
sing devices. 

Yet another object is to provide a means for mairk- 
ing tasks as having affinities to particular prooessore 
45 of a network, or alternatively as having an afHnlties for 
all such processors, to redistribute teste across the 
network when desired. 

To achieve these and other objects, there is pro- 
vided a process for associating each one of a plurality 
so of date operations with one of a plurality of processing 
devices, for subsequent execution of the operation by 
the associated processing device. The process is per- 
fonned on an information processing network includ- 
ing a nrain menrwry having nnemory locations for 
55 storing bit-encoded data, a plurality of processing 
devices for manipulating bit-encoded data, and a 
main storage intertece connected to the prooessore 
and to the main memory. Each d the processing 
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devices has a cache memory including a plurality of 
cache locations for storing bit-encoded data. Each 
processor further includes means for storing and ret- 
rieving bit*encoded data to and from the main storage 
memory via the main storage internee, as well as s 
means for storing and retrieving bit-encoded data to 
and flrom its associated cache memory. The t)it-enco- 
ded data tndudes data operations, each of which has 
priority indicia, and further has affinity Indicia to indi- 
cate ^ther a general afRnify to more than one of the to 
processing devices, or a specific affinity to a selected 
one of the processing devices. 

AOocation includes the following steps: 

(a) performing a primary scan of the data oper- 
ations, preferably In an order of descending is 
priority beginning with the highest priority data 
operation, including scanning the affinity indicia 

of each data operation for the presence of a 
specific affinity; 

(b) for each data operation having a specific 20 
affinity for a selected one of the processing 
devices, either (i) assigning the data operation to 

the selected processing device if the processing 
device is available for the assignment, or (ii) 
bypassing the data operation in the event that a 2S 
higher priority data operation has been ass^ned 
to the selected processing device; 

(c) in the event that the data operation has a gen- 
eral affinity, proceeding to the data operation hav- 
ing the next highest priority; 30 

(d) continuing steps (b) and (c) until each one of 
the processing devices has an associated one of 
the data operations assigned to it; and 

(e) dispatching each assigned operation to as 
associated processing device. 35 
Preferat>!y, the process further includes setting a 

bypass threshold. After bypassing a number of data 
operations equal to the bypass threshold, each sub- 
sequentty scanned data operation is assigned to one 
of the processing devices to %vhbh a data operation 40 
has not yet been assigned. In other words, any data 
operation having specific afRnity for a processing 
device that was already assigned to a higher priority 

data operation is treated as if it had general alfDiiy as 

in step (c). 45 

Further, upon detecting a generd affinity as in 
step (c) atiove, it is preferable to reserve each data 
operation having a general affinity for a sui)sequent 
assignment to one of the processing devices. Then, 
when the number of reserved data operations equ^s so 
the number of processing devices still avaaabie, a 
secondary scan Is perfomied upon previously scan- 
ned data operations. Then, each previously reserved 
data operation is assigned to one of the remaining 
available proces^ng devices. 55 

Afurtherstep of the process can be utilized in the 
event that after the secondary scan and assignment 
of all remaining reserved processing devices, one or 



more processing devices remains available while 
unasslgned data operations remain, having been 
bypassed according to step<b} above. Then, the pre- 
viously bypassed data operations are assigned, dis- 
regarding tiieir specific affinity, to the next available 
proosssor, until each tasic has t>een assigned a pro- 
cessor. The secondary scan, of previously reserved 
as weO as previously scanned data operations, pref- 
erably is performed in the reverse orderof the primary 
scan, Ib. in order of ascending priority. 

As the data operations are dispatched to their 
associated processors, tiie affintty indicia for each 
operation preferaBiy are modified to identity each 
operation or tasIc as having a specific affinity to the 
processor to which it is dispatched. L^ter, when a tasfc 
is switched out of its associated processor, it retains 
Us specific affinity; When reassigned, ttie tasic is more 
likety to be assigned to ttie associated processoronoe 
again. 

Another feature of the present invention resides 
in a means for either conditionalty or unconditionally 
modifying ttie indicia of tasi® from a specific affinity to 
a general afRnity. More particuiariy, each task or oper- 
ation conBsponds to a task dispatching element 
(TDE) that includes an affinity field. Each affinity fiekl 
is set by entering a binary word identfying a particular 
processing device. However, when a certain task is 
switched out of a processor, ft may not return for 
further execution for a long time. This raises the likdi- 
hood that the cache memory no longer wil be *hor in 
flie sense of retaining data and instnictions pertment 
to that task upon iis return. In this event, instructions 
associated with the task reset the TDE affinity field to 
a general affinity for all of the processing devices, or 
altemativety may reset ttie affinity field upon a fsSUm 
to satisfy a predetennined condition. The uncon- 
ditional reset instrMCtion is prefierably used in connec- 
tion with tasks of a high priority, in order to avokl 
preemption of such tasks by k>w^ priority tasks. 

In accordance witti ttie present invention, a specie 
fic affinity nonnaliy is assigned to each data operation 
or task, tiius substantialiy increasing the likeiihooc 
that such task, when reassigned to one of a plurality 
of processore, is associated witti the processor tfiac 
previously executed ttie ta^ This substantialty 
increases ttie numl>er and proportion of tasks exec- 
uted using the cache memory as opposed to the main 
storage memory. Increased use of the cache 
niemories enhances network effteienc^, since gaining 
access to data from one of the cache memories con- 
sumes less ti'me than accessing the same data from 
main storage. OveraO processing networic efficient 
is thus increased/ 

A salient feature of the present invention is that 
such gains in efficiency are achieved witiiout sacrifi- 
cing ttie abi'ity to set Ngher priorities for relativety 
important tasl^ and without calling or pennitfif^ an 
imbalance of worfc load among the various processing 
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devices. More particularty. the option to reset pro- 
cessor affin'ty for selected taslcs, in combination with 
reserving the tasks with general affinity during the 
primary scan, ensures that such tasks are not subject 
to excessive delay from the assignment of lower 
priority tasks with specific processor affinity. When 
previously reserved and In some cases previously 
bypassed tasks are assigned during the secondary 
scan and without regard to processor specific affmlfy. 
the result is a substantially equal distribution of work 
load among the processors. Thus, in accordance with 
the present invention, a compromise or balance is 
achieved among competing objectives of high cache 
usage, favoring high priority tasks and substantially 
equal distribution of work load among processing 
devices. 

For a further understanding of the above and 
other features and advantages, reference is made to 
the following detailed description and to the drawings, 
in which: 

F*^. 1 1s schematic view of an information proces- 
sing system in which multiple processing devices 
share main storage memory through a common 
interface; 

F^. 2 is a representation of a chain of task objects 
or locatlols, each object associated with and iden- 
tifying one of a plurality of tasks or data oper- 
ations; 

Fq. 3 is a representation of one of the task 
objects; 

Ftgs. 4-6 are sections of a flow chart illustrating a 
process for assigning task objects from the chain 
for dispatching to associated processing devices; 
and 

Figs. 7 through 10 respectively Qlustrate the pro- 
cess as perfomned under various conditions. 
To focilitate an understanding of the prefenred 
embodiment, this descriptkin begins with a brief exp- 
lanation of certain tennns used fe descrit)e the oper- 
ation and features of the invention. 

IMPI Instructions: Assembly level computer pro- 
gram Instiucttons In mkmcode. Processing devices 
execute IMPI instructtons sequentially in perfbnming 
logical operations on data, e.g. store, add, and move- 
character functions. 

IMPI Procedure: An IMPI lnstructk>n stream. 
Including the data used by the Instruction stream, and 
the parameters and other Infonnation necessary for 
transferring Information to or from other IMPI proced^ 
ures. 

IMPI Task (or Data Operation): A synchronous 
execution of one or more IMPI procedures, compris- 
ing a unit of wortc executed or executable by a proces- 
sing device. 

Horteontal Microcode (HMC): Mtorocode that 
Interprets IMPI instructions. I.e. decomposes the IMPI 
instructions into a series of simple steps. HMC 
instructions also are caDed control words, and govern 



the state of the hardware. 

Cache Memory: This emt>odffnent concerns 
cache memories of the one-way associative write- 
thrxnjgh type. Each cache memory holds t>oth data 
5 and IMPI instructions. 

Natural Affinity: A task or data operation has a 
natural affinity for a processor, if the cache memory of 
that processing device contains data and Instructions 
pertinent to that task. Natural affinity tends to increase 
to as the processing device continues to execute the 
task, and otherwise tends to decay. 

Turning now to the drawings, there is shown in 
Fig. 1 an information processing network 16 for stor- 
ing and performing various operations upon bit-enco- 
16 ded data. The network in this example Includes four 
processing devices indicated 18, 20, 22 and 24, but 
more or fewer processing devices could t>e used. An 
interfece 26 couples the processing devices to a main 
storage memory 28. 
20 Processor 18 includes a control store 30 where 

the horizontal microcode (HMC) resides. A portion of 
the horizontal microcode Is a dispatching program 32 
used in allocating various data operations or IMPI 
tasks for subsequent execution or performance on 
25 one of the processing devices. The processor further 
includes an array of general purpose IMPI registers 
34. When an IMP! task is switched into processor 18 
for execution, data and IMPI instructions pertinent to 
the task are loaded into registers 34 either from matn 
30 storage memory 28 or from the processor's cache 
memory 36. When the task is switched out of pro- 
cessor 18, processor data Is stored to the main stor- 
age memory, and also may be stored to the cache 
memory, if the associated cache lines are valid. 
35 Processing device 18 further includes a one-way 

associative store-through cache memory 36, which 
includes a plurality of cache lines, each assodaled 
with a particular set of addresses or locatiorts in the 
main storage memory. Cache memciy 36 further 
40 includes a cache directory 38. The cache directory Is 
a record of the main storage memory addresses map- 
ped into the cache mennory, and stores an indication 
of whether each ooiresponding location in the cache 
memory is valid or invalid. The directory also contains 
45 status bits for each cache line and its associated 
address in main storage. As indicate at 40, pro- 
cessor 1 8 includes logic (In the fonm of semiconductor 
chip circuitry) for fetching and storing data, and for 
perfbmning arithmetic logto operations upon blt-enoo- 
50 ded data. A portion of logic 40 Is dedicated to control- 
ling stores of data to cache memory 36, and fetches 
or retrieval of data from the cache merrtory, via a 
cache Interface 42. 

Arbitration links at 44 Interconnect the four pro- 
55 cessing devices and are used in combination wfth 
art)ltratlon logic residing in the processing devtoes to 
assign priority to any one of the processors. In terms 
of access to Interfece 26 l>etween the prooessora and 
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main storage. 

Processing devices 20. 22 and 24 are similar to 
processing device 18, and include respective oontrof 
stores 46, 48 and 50; respective dispatehing prog- 
ranris 52, 54 and 56; and respective IMP! registers 58. s 
60 and 62. All of dispatching programs 32, 52, 54 and 
56 are identical to one another, so that any one of the 
processing devices can allocate tasks to all of the pro- 
cessing devices, including itself. Each of the proces- 
sing devices has the same ability to gain control of the io 
program, although only one of the processing devices 
controls the dispatching program at any given tinte. 

Processing devices 20, 22 and 24 are further like 
processor 18 in having respective cache memories 
64, 66 and 68; respective cache directories 70. 72 and is 
74; respective store and fetoh logb 76, 78 and 80; and 
respective cache interfaces 82, 84 and 86. -These 
components function in substantially the same man- 
ner as their counterparts in processing device 18. 

Main storage memory 28 includes memory arrays 20 
88 for storing bit-encoded data, and a program 
instruction store gO in which the IMPl instructions are 
embedded. Also in main storage is a task dispatching 
queue (TDQ) 92 including a TDQ header 94 and a 
plural!^ of task dispatching dements (TDEs) 96-106, 2S 
an^nged in a chain from the highest to the lowest 
priority. Preferal>iy, main storage memory 28 is conv 
prised of multiple individual memory cards, each hav- 
ing its own memory arrays and its own instructktn 
store segment the segments cooperating to provkie so 
the IMPl instructions. Various memory arrays cooper- 
ate to provide the chain of TDEs. 

A task-dispatohing scratch pad memory 108 in 
main storage is used for temporarily reteining infor- 
mation Ireful in allocating tesks. including: (1) an 3S 
Assigned Processor Count of processors to which a 
task has been assigned; (2) the identification of assig- 
ned processors; (3) an Assigned Task Count; a Dis- 
patchabie Task Count of the number of tasks either 
assigned or reserved for later assignment; (4) a 40 
record of which processing devices remain avaOable; 
(5) a Scanned Task Count; and (6) a Skip Count of 
bypassed TDEs. 

Interface 26 can include a plurality of separate 
busses (not shown) in parallel, each shared in com- 4S 
mon by all of tiie processing devices and the memory 
cards of main storage. For example, working date (the 
informatton of most direct concern to users of the net- 
work) is transmitted over a working date bus. A conv 
mand/address bus transmite information controiiing so 
the transmissions of woricing date, including infor- 
mation as to the address in main storage memory 28 
at which woiknig date is to be stored or retrieved. A 
communication bus transmits the statos of working 
date as the working date is transmitted via the date 55 
bus. Date nnes 110. 112, 114 and 116 connect pro- 
cessing devices 18, 20, 22 and 24 to interface 26. A 
date line 118 connecte the interface to main storage 



memory 28, altiiough in tiie case of multiple memory 
cards, each card of course has Ite own date line to the 
interface. 

Tasks or date operations eligible for execution by 
processors 18-24 (i.e. by any one of the prooessore) 
are allocated to the processors through the task dis- 
patching program, by which a primary scan is perfor- 
med to detennine the processor afTmity for eac^ task. 
For scanning, tiie task dispatching elemente identify- 
ing each of the tasks are arranged in task dispatching 
queue 92. As seen ki F*q. 2, header 94 of ttie TDQ 
includes a pointer field 120 and a tiireshold fleid 122. 
The remainder of the TDQ is comprised of tesk dis- 
patehing elements 96-106. Pointerfieid 120 locates or 
"points to" tiie highest priori^ task, as identified by ite 
corresponding TDE 96. Threshold field 122 contains 
a value representing the maximum number of task 
dispatching elements to be bypassed during the prim- 
ary scan, as is explained in greater deta9 below. 

^ The fonmat of tesk dispatching element 96 is 
shown in F*^. 3, it being understood thattiie remaining 
TDEs have substentially the same format TDE 96 
includes a priority field 124 oonteinmg a binary value 
indicating the prkuify or importence of the associated 
tesk. In tills instance, the lower the value in field 124. 
the higher tiie priori^ of the comesponding tesk. An 
IMPl queuing fonction operates on respective priorily 
fields of tesk dispatching elemente 96-106, to arrange 
the TDEs in a chain of descending priority, i.e. of 
increasing value in tiie priority field. Should two or 
more of TDEs 96-106 have the same priori^ value, 
tile task later inserted Into tiie TDQ is given tiie lower 
priority focation akmgihe chain. Task dispatching pro- 
gram 32, 52, 54 or 56 (depending on ttie controlling 
processor) scans tiie task dispatching elemente in tiie 
order In which tii^ appear in ttie chain, resulting in 
scanning the tasks in an order of descending priority 
based on the position of each TDE wittiin the TDQ, 
and indirectiy based upon the priority field values. 

Each task dispatching element further includes a 
processor afFinily field, as indicated at 126 for TDE 96. 
In connection vritii a four processor network as des- 
cn*bed, affinity field 126 foidudes four bit posttkms, a 
binary "1 " in a particular bit position indicating a sp^ 
fic affinity, Le. alfinit^rfor only one of the processors. 
A binary "1 ' In all of ^ bit positions indteates a gen- 
eral affinity, i.e. affinityof theteskforallfourof the pro- 
cessors. Thus, ttie pattern "01 00* In field 126 
indicates tiiat ttie task associated with TDE 96 has an 
affinity for processing device 20. 

The tesk dispatching element includes twofurtiier 
fields, namely a pointer fieW 128 and an end-of-ctiatn 
field 1 30. Pointerfieid 128 functions in a manner simi- 
lar to pointer 120 of tiie TDQ header, pdnting to or 
locating the task dispatehing element of next priorfty, 
to provkie a "IMc" in ttie TDE chain. The end of chain 
field provides at lea^ one bit position for a birmry flag 
virtiich, when present, indicates ttiat ite associated 
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task dispatching element is the final or lowest priority 
TDE in the task dispatching queue, thus signifying the 
end of the TDE chain. It is to be understood that the 
TDEs contain other data fields not directly concerned 
with the present Invention, and thus not described. 5 

Figs. 4-6 are sections of a flow chart useful in 
understanding the manner in which the task dispatch- 
ing program and task dispatching queue are used to 
allocate tasks to the processing devices. In general, 
the TDEs are scanned in order of descending priority, io 
and either assigned to one of the processors, bypas- 
sed, or reserved for subsequent assignment. After the 
primary scan, a secondary scan is performed if neces- 
sary to assign any previously reserved tasks to any 
processing devices not yet associated with a particu- is 
iar task at the end of the primary scan. If necessary, 
previously bypassed tasks are assigned to any pro- 
cessors not yet associated, to ensure that to the 
extent possible a task is dispatched to each of the pro- 
cessor. 2^ 

Before the primary scan, IMPI instructions 
initialce the contents of scratch pad memory 108 to 
indicate that all of the processing devices are avaO- 
able. The scratch pad memory further is initialized to 
indicate that none of the tasks has been assigned, 25 
none of the ta^ has haen either reserved or assig- 
ned, none of the tasks has been bypassed, and none 
of the tasks has been scanned. 

The highest priority task dispatc^iiilg element is 
considered fcrst, as indicated at 1 32. Before the scan, so 
there is a detemtination at 134 of whether the previ- 
ous TDE scan was the end-of-chain or lowest prbrity 
TDE, Le. whether a flag was detected in the previous 
endKtf-chain field 130. If so, there Is no^cunentTDE'' 
and the process proceeds to the reverse scan. If not, 
the accumulated number of dispatchable tasks (Dis- 
patchable Task Count) is compared to the number of 
processors at 136. Once the Dispatchable Task 
Count equals the number of processors, the process 
proceeds to reverse scan. 

If the Dispatchable Task Count does notyet equal 
the number of processing devices, the affinity field of 
the current TDE Is scanned. The affinity field of each 
TDE s scanned at 138 for the presence of a spedfic 
affinity Indicating one of processing devices 1 8, 20, 22 
and 24; or a general affinity. Hw. general affinity ref- 
ers to having affinity for all four of the processing 
devices, although in a broader sense, general affinity 
could indicate affinity for any two or more of the pro- 
cessus. If the TDE has a general affinity, then an 
appropriate field m scratch pad memory 108 is set to 
a value *'an/' to indicate that the task Is going to be 
assigned to a processor, but the particular processor 
has not yet been determined. The Dispatchable Task 
Countand Scanned Task Count are Incremented, and 
the process r^ums to the deciston at 134foreithera 
scan of the next TDE era shift to reverse scan as pre- 
vtousiy explained. 



Alternatively, if the scan shows the current TDE 
has a specific affinity at 1 38, scratch pad memory 108 
is searched to detenmine whether the selected pro- 
cessor remains available, i.e. has not yet been assig- 
ned to another, higher priority task. If the selected 
processing device is available, the scratch pad mem- 
ory is modified to show the processor as associated 
with the current task, and the cunrent task dispatching 
element is assigned to the selected processor. When 
the current TDE with a specific affinity is assigned, the 
Assigned Processor Count in scratch pad memory 
1 08 is incremented. The identity of the processor also 
recorded In the scratch pad memory. As before, the 
Dispatchable Task Count and Scanned Task Count 
are increrr^nted. 

Returning to decision 138, the scratch pad mem- 
ory search may yield the result that the selected pro- 
cessing device is not available. In this event the 
accumulated total of bypassed TDEs (Skip Count) is 
compared to the threshold value in field 122 of TDQ 
header 94. If the Skip Count is less than the threshold, 
the current task is identified as bypassed or skipped 
in the scratch pad memory, and the Skip Count and 
the Scanned Task Count are incremented. The Dis- 
patchable Task Count is not incremented. 

Returning to deciston 1 38, a final possibility is that 
a selected processing device is not available to the 
current task, but the bypass threshold has been 
reached. In this event, the specific affinity of the TDE 
is ignored, and scratch pad memory 108 ^ modified 
to indtoate that the task will be assigned, although ths 
specific processing device Is not yet determined. 
Thus, the current TDE is treated as if it originally had 
general affirtity. It is not yet assigned, but mserved for 
later assignment 

In any event, either the primary scan completes 
the chain of task dispatching elements, or the number 
of tasks either assigned or reserved t>eoomes equal 
to the numt>er of processors, and allocation proceeds 
to the reverse scan stage. 

At the t>eginning of the reverse scan, the Assig- 
ned Task Count is compared with a "usable" proces- 
sing device count (Usable Count) equal to the lesser 
of (0 the number of processing devices and (ii) the 
number of tasks in the task dispatching queue. If, in 
view of the comparison at 140, the Ass'^ned Task 
Count equals the Usable Count, the allocation of 
tasks is complete, and the tasks are dispatched, each 
to its associated one of processlr^ devices 18-24. So 
long as the number of tasks assigned is less than the 
usable deslgnatton, at least one processing device 
remains avaflable. and at least one task In the TDQ Is 
not yet assigned. 

The secondary scan t>egins with the lowest 
priority task at the end of the chain and proceeds in 
an order of ascending priority. 

In the secondary search, scratch pad memory 
1 08 Is searched in connectton with the cunrent TDE for 



40 
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the value of "any" in the processor field, indicating that 
the current TDE was previously reserved but not 
assigned. If such is the case, the current task is assig- 
ned to the next available processor and the processor 
is identified in the scratch pad memory as associated s 
with the current task and therefore no ionger avail- 
able. The count of assigned processors is incremen- 
ted at 142, and the scan pn^ceeds to the next, higher 
priority TDE. The scan proceeds in this manner for all 
reserved TDEs, whether reserved because of an io 
original general affinity or because the bypass 
threshold was reached. 

In cases where the sum of the number of proces- 
sing devices and the bypass threshold exceeds the 
number of task dispatching elements in the TDQ, is 
there is the possil>ility of at least one of the processing 
devices remaining available, along with one or more 
previously bypassed tasks remaining unassigned. 
This situation is addressed when the scratch pad 
memory search for the cunnent TDE fails to find the 20 
value "an/". Then, the scratch pad memory is ftjrther 
searched for the "skip" indication corresponding to the 
current TDE. If the skip indicatk>n is notfound, the cur- 
rent TDE already Is assigned, and the scan proceeds 
to the next task dispatching element 25 

If the skip indication is found, however, and the 
Dispatchable Task Count remain less than the Usable 
Count the Dispatchable Task Count is incremented, 
and the current TDE is assigned to the next avaiatile 
processing device, with such processing device then so 
identified in scratch pad memory 108 as no longer 
availabia The Assigned Processor Count is incre- 
mented, and the scan proceeds to the next task dis- 
patching element The reverse scan proceeds until 
the count of assigned processing devices equals the 35 
Usable Count at which point the allocation process is 
conrq>lete, and tasks are dispatched to their respective 
processing devices. 

Each of Rgs. 7-1 1 is an example of how a plural ity 
of tasks or data operations, chained in task dispatch- 40 
ing queue 92, are allocated in accordance with the 
primary and secondary scans. In each case, the task 
dispatehtng elements are kientified individually as A, 
B, C, eta, in order of descencfing prforfty. The afRnity 
of each task at the t>eginning of the process is listed, 45 
Indicating one of processing devices 18-24 or "ALL" 
forataskwith general affinity for all of the processois. 
it is assumed that each of the tasks is el^ible for 
execution on any one of the four processing devices. 
Finally, the bypass tfireshold or skip number s shovm so 
at the top of each table. 

Rg. 7 niusbates a basic example of four tasks, 
each with a specific affinity for a different one of the 
processing devices. The tasks are assigned directly 
and in the order shown. ss 

In Rg. 8, task A is signed inrunedfeitely but task 
B Is bypassed. Task C is reserved (its affin fty changed 
to "ALL*), since the bypass threshold has been 



reached. Likewise, task O is reserved. Task E is 
immed^tely assigned to processor 20. Then, during 
the secondary scan, task D is assigned to processor 
22 and taskC is assigned to processor 24. Task B is 
not allocated until a sut>sequent scan, usually as part 
of a succeeding execution of the task dispatohar. 
Thus, In a case where altocatlon based sdely on 
afHnify would cause a severe load imbalance and 
leave two processors idle, the reverse scan assures 
that all of the processors remain busy. 

In Rg. 9, tasks A, C and E have a general affinity 
and thus are reserved during the primary scan, task 
B is assigned to the processing device 1 8 for which it 
has specific afUnify. Task D is bypassed. During tfie 
reverse scan, tasks E, C and A are assigned to pro- 
cessing devices 20, 22 and 24, respectively. 

In Fig. 10, all of the tasks have an affinity for prx>- 
cessor 1 8. Task A is assigned immediately, and tesks 
B and C are bypassed. Tasks D and E are reserved, 
in view of reaching the bypass threshold. In the 
reverse scan, task E is assigned to processor 20, task 
D is assigned to processor 22, and taskC, prevrousty 
bypassed, is assigned to processor 24. 

Finally in Rg. 10, all tasks except task B (general 
affinity) have an aflRnity for processor 22. Task A is 
assigned immediatety, task B is reserved, and tasks 
C and D are skipped. The threshold being reached, 
task E is reserved, in the reverse scan, task E is assig- 
ned to processor 18. p^e^nously skipped task D is 
assigned to processor 20, and task B is assigned to 
processor 24. 

Thus, a primary scan based on processoraffinl^. 
In combination with a secondary scan of the same 
tasks but in reverse order, stnlces a balance among 
desirable but sometimes competing ob^ctives of 
maximizing usage of cache memory, Coring tasks of 
h^h priori^, and keepnig all available processors 
busy. In connection with nehA^ortc 16 having four pro- 
cessing devices, a bypass threshold of eittier one or 
two has been found most satisfactory. It is to be 
understood that beyond the number of processing 
devtoes involved, the natore of the tasks being exec- 
uted and other fiactors also will influence selection of 
the bypass threshold. 

One particularfy useful feature of ttie present 
invention is the (fynsmSc assignment of specific 
affinity to a gh/en task. More particularly, as a task, 
prevtously allocated to a selected processor, is 
switohed into that processor, the affinl^ Held of the 
corresponding TDE is set to identify that prvxsessing 
devtee. When tiie task is switohed out of ttie 
associated processor, ttie processor identification 
remains in tfie affinity Held. Thus, the next time the 
task is to be aDocated, ttie corresponding TDE «hi- 
btts affinity for the selected processor, to maximize 
the probability of reassignment to the same prooes- 
sing device. 

In some cases, however, it is advantegeous to 
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prevent the othenAfise aulomatic assignment of pro- 
cessor affinity to a given task. For example, certain 
IMPI tasks may be identified as having such a high 
priority that they should never be bypassed during the 
preliminary scan. Similariy, general affinity for certain s 
high priority tasks might be deemed preferable to the 
risk of severad high priority tasks having the same 
specific affinity, to avokJ a situation of high priority 
tasks being required to wait while tasks of lower 
priority are run on other processing devices. f o 

To these ends, a spedal instruction deemed 
Reset Processor Affinity (RPA), is induded in any 
chosen IMPI procedure or task. The RPA instruction 
unconditionally resets the assigned affinity, i.e. 
inserts a general affinity des'ignatton "1111** into the i5 
processor affviity field of the corresponding TDE in 
lieu of the processor klentification. 

Further in accordance with the present invention, 
means are provided for resetting the processor affinity 
field for general affinity, if a certain predetennined 20 
condition is not met More particulariy, network 16 
utilizes certain °8end° and "receive" messages in IMPI 
instructions to synchronize separate IMPI tasks. For 
example, a task A, executing on processing device 
1 8, can not be completed until a task 6, executing on 25 
processor 20, reaches a predetermined stage. Pro* 
cesser 20 issues a send message when task B 
reaches the predetermined stage. Meanwhile, pro- 
cessor 18 issues a receive instructk>n, seeking the 
send message from processor 20, 1.6. seeking the 30 
indication that task B has reached the predetermined 
stage. 

If task B has not reached the predetermined stage 
by the time processor 18 provides the receive mes- 
sage, the receh^e instmctlon is deemed "not satis- 35 
tied". Task A is removed from the task dispatching 
queue, and task dispatching program 32 switches out 
task A. Task A then waits for reallocation and dis- 
patching. If the warit Is suffidently long, task A loses 
its natural affinity for processor 18. 40 

In situations where the waiting period for task A 
is expected to be sufficiently long to decay niatural 
affinity, a conditional instructton is induded in the IMPI 
task, spedfic^ly to reset the affinity field in the event 
that the receh^ message is not satisfied. Thus, seleo- 4S 
ted IMPI tasks can be configured for general affinity 
upon being switched out of one of the processing 
devices, either unconditionally or subject to an event 
such as a receive instruction not being satisfied. In 
combinatton with the prevbusly described primary so 
and secondary scans, this further ensures ttiat task 
aOocatton responsh/e to assigned affinity does not 
unduly diminish preference for high priority tasks, or 
efficiency in workload distribution airrang processors. 



Claims 

1. In an information proces^ng networi^ induding a 
main memory having memoiy locations for stor- 
ing bit-encoded data, a plurality of processing 
devices for manipulating bit-encoded data, and a 
main storage interface connected to the proces- 
sing devices and to the main memory; each of the 
processing devices having a cache memory 
including a plurality of cache locations for storing 
bit-encoded data, each processing device further 
including means for storing and retrieving bit-en- 
coded data to and from the main storage memory 
via the main storage interfece and means for stor- 
ing and retrieving bit-encoded data to and fit)m Its 
associated cache memory; said bit-encoded data 
including data operations, each of the data oper- 
ations having priority indteia and having affinity 
indk:ia to indicate either a general affmity for at 
least two of the processing devices or a spedfks 
affinity for a selected one of the processing 
devices; a process for associating each of the 
data operations with one of the processing 
devices for subsequent execution of the oper- 
ation by the associated processing devtee, 
including the steps of: 

(a) performing a primary scan of the data oper- 
ations, induding scanning the affinity Indtoia 
of each data operation for the presence of a 
specific affinity; 

(b) for each of the data operations having a 
specify affinity for a selected one of the pro- 
cessing devices, either (i) assigning the data 
operation to the selected processing device if 
the processing device is available for such 
assignment or (si) bypassing the data oper- 
ation in the event that a higher priority data 
operation has t>een assigned to the selected 
processing device; 

(c) in the event that the data operation has a 
general affinity for the processing devices, 
proceeding to the data operatkm having the 
next highest priority; 

(d) perfomning steps b and c until either all of 
the data operations have been assigned* or 
until each one of the processing devices has 
an associated one of tfie data operations 
assigned to it; and 

(e) dispatching each assigned operation to its 
associated processing device. 

2. The process of Q^m 1 wherein: 

said general affinity Is an affinity for all of 
the processing devtees. 

3. The prooessof aaim 1 wherein said primary scan 
Is performed In order of descending priority of 
said data operations. t>eglnning with the one of 
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said data operations having the highest priority. 



order of ascending priority of the data operations. 



4. The process of Claim 1 including the further step 
of: 

setting a bypass threshold prior to said s 
primary scan and, after bypassing a number of 
the data operations equal to the bypass 
ttveshold, assigning each of the subsequently 
scanned data operaticvis to any one of the pro- 
cessing devices to which a data operation has not io 
yet been assigned, disregarding the specific 
affinity, if any, of said sutTsequently scanned 
operations, until one of the data operations has 
been assigned to each one of the processing 
devices. 15 

5. The process of Claim 4 wherein said subse- 
quently scanned data operations having specific 
affinity for a processing device that was already 
assigned to another data operation is assigned a 20 
general affinity. 

6. The process of Claim 1 including the further steps 
of: 

upon determining no specific affinity for a 25 
data operation being scanned, reserving the data 
operation for subsequent assignment to one of 
the processing devices; 

counting the number of said data oper- 
afions reserved for subsequent assignment and, 30 
responsive to detemiinlng that the number of 
reserved data operations equals the number of 
said processing devices to which no data oper- 
ation has t>een assigned pursuant to step b, per- 
fomting a secondary scan upon the previously 3S 
scanned data operations; and 

for each of the data operations reserved 
for sut)sequent assignment assigning each ssdd 
reserved data operations to one of the remaining 
available processing devices. 40 

7. The process of Claim 6 Including the further step 

following said assigning of each reserved 
data operation, and upon detenmlning that at least 4S 
one of the processing devices remains avaiabte 
for assignment of a data operation and that at 
least one of the previously scanned data oper- 
ations was bypassed, further perfonming the sec- 
ondary scan on each said bypassed data so 
operation and assigning it to an avaQable proces- 
sing device, disregarding the specific affinity, untS 
eitfier (i) all of the processing devices tiave been 
assigned one of the data operations; or (ii) all of 
the data operations have been assigned. ss 

8. The process of Claim 6 wherein: 

said secondary scan is performed in an 

10 



9. The process of Claim 8 wherein: 

said step of dispatching the data oper- 
ations further indudes modifying the affinity indi- 
cia of each dispatched data operation to exhibit 
the specific afHnlty for the processing device to 
which it is dispatched. 

10. The process of Claim 8 wherein: 

the specific affinity, of each one of the data 
operations having a specific affinity, is based 
upon the one of said processing devices that 
most recently executed the data operation. 

11. An infomiatlon processing network including a 
main memory having memory locations for stor- 
ing bit-encoded data, a plurality of processing 
devices, and a main storage interface connected 
to the processing devices and to the main mem- 
ory; 

each of the processing devices having a 
cache memory including a plurality of cache loca- 
tions for storing bit-encoded data, each proces- 
sing device further including means for storing 
and retrieving bit-encoded data to and from the 
main storage memory via the main storage inter- 
face, and means forstoring and retrieving bit-en- 
coded data to and from its associated cache 
memory; 

said blt-erux>ded data including data oper- 
ations comprising woridng data and program 
instructions, said processing devices using 
instruction streams of the program instructions to 
manipulate the worlcing data, each said instruc- 
tion stream comprlsnig a pluredity of the program 
insbucttons; 

wherein tlie improvement comprises: 

processor affinity indicia associated with 
each of the data operations and identifying the 
associated data operation as having eithera gen- 
eral affinity for at least two of the processing 
devices, ora specific affinity for a selected one of 
the processing devices; 

data operation altocation means for scan- 
ning a plurality of the data operations, and for 
associating each of the data operations with one 
of the processing device for subsequent execu- 
tion of the instruction stream by the associated 
processing device, the associating of each date 
operation l>eing t>ased at least in part upon the 
affinity indicia of the data op^ation; 

a means for modifying the processor 
affinity Indicia of each date operation, upcm the 
associating of ttie date operation wltti one of the 
processing devices, to exhibit the specific affinity 
for the associated processing device; and 

at least one processor afRnity reset 
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means* each affinity reset means being 
associated with a selected one of the instruction 
streams, for resetting the processor affinify indi- 
cia of the associated selected data operation to 
exhibit said general affinity as the associated pro- 5 
cessing device executes the selected instruction 
stream of the data operation. 

12. The information processing netwoik of Claim 11 
wherein: 

each of the data operatons includes a pro- 
cessor affinity field, and the processor affinity 
Indicia of each data operation include a binary 
word in the affinity field. 

IS 

1 3w The processing network of Claim 1 2 wherein: 

said reset means includes a reset instruc- 
tion of tiie program instructions, the reset Instruc- 
tion being inserted into each of the selected 
Instruction streams, said reset instruction, when 20 
its associated selected instruction stream is exec- 
uted by the associated processing device, reset- 
ting the binary word in the associated processor 
affinity field to a value corresponding to eaid ger>- 
erai affinity. ^ 

14* The processir^ networic of Claim 13 wherein: 

said program instructions include a 
receive instruction and a send instruction in fir^i 
and second ones of the InstnicQon streams, re- so 
spectively, for syndvonlzing the first and second 
data operations; wherein the receive instruction 
when executed seelcs a message that a predeter- 
mined condition is satisfied, and the send instruct 
tion when executed provides a message that the S5 
condition is satisfied: and wherein the Instruction 
stream of the first data operation, when executed, 
continues to execute if said message Is detected 
when the receive Instruction is executed, and the 
data operation Is switched out of the associated 40 
processing device if the message is not detected 
when the receive instruction is executed; and 

wherein the processor affinity field reset 
means includes a conditional reset means, 
ffisodated witii tiie instruction stream of tiie first 4$ 
data operation, for maintaining the specific affinity 
of the first data operation whenever ttie message 
is so detected, and for resetting the processor 
affini^ field of the first data operation when ttie 
message is not so d^ected. ^ 

15. The infdnration prcx:essing network of Claim 14 
whmin: 

said conditional reset means comprises a 
binary flag witiiin a field of the receive Instruction. 55 
operable to cause said resetting only upon the 
failure to detect said message wtien ttie receive 
instruction is executed. 

11 
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(g) In connection with an information processing 
network (16) in which multiple processing de- 
vices (18,20,22,24) have individual cache 
mennories and also share a main storage mem- 
ory(28), a process is disclosed for allocating 
multiple data operations or tasics for subse- 
quent execution by the processing devices. A 
plurality of task dispatching elements (TDE) 
(96-106) forming a task dispatching queue 
(TDQ) 92 are scanned In an order of descending 
priority, for either a specific affinity to a selected 
one of the processing devices, or a genera! 
affinify to all of the processing devices. TDEs 
with specific affinity are assigned immediately if 
the selected processor is available, while TDEs 
of generai. affinity are reserved. TDEs wHh a 
specific affinity are bypassed if the selected 
processor is not avaSable. or reserved if a pre- 
detenmined bypass threshold has been reached. 
Following the primary scan a secondary scan, in 
an order of ascending priority, assigns any 
reserved tasks to the processing devices still 
available, without regard to processor affinity. 
Previously bypassed tasks can t>e assigned as 
wefl, in the event that any processor remains 
available. A further feature of the network is a 
means to reset the processor affinity of a selec- 
ted task from the specify affinity to the general 
affinity. Resetting is accomplished through an 
assemt>ly level instruction contained in the task, 
and eittier can t>e unconditional, with reset 
occurring whenever the task is executed on one 



of the processing devices, or can occur only 
upon the faflure to meet a predetermined condi- 
tion whae the task is executing. 
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